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1. Introduction 

The text for my sermon today is the closing paragraph of Hme's 
Enquiries Concerning Human Understanding , 

When ve run over libraries, persuaded of these principles 
(Hume's principles of the understanding), what havoc must we 
make? If we take in our hand any volume; of divinity or school 
metaphysics, for instance; let us ask. Does it contain any 
abstract reasoning concerning quantity or number ? No. Does 
it contain anjr experimental reasoning concerning matter of 
^^^^ and existence ? No. Commit it then to the flames: for 
it can contain nothing but sophistry and illusion. 
Hume would be the first to admit that we are all entitled to our' fantasies, 
but he would also insist that we recognize them as the fantasies they are. 
To reformulate his text, general ideas about educational policy and prac- 
tice contain little but sophistry and illusion, unless they can be defended 
by abstract reasoning from some other accepted general principles or be 
inferred in a definite manner from particular matters of fact. 

This sounds like a hard line, and it is. It is too hard to be used 
at all times and places in discussions of educational policy and practice, 
but it is not too hard for a reflective occasion like this one, which is 
aimed at appraising the relevance of research to educational ideas. 
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Rather than begin with any general remarks or general propositions, 
I shall first report some of the results of perusing my own library and 
applying Hume's tests. The initial examples that I critically examine 
will be those that most of us can sagely and benignly agree about. I 
shall move on to others that are more controversial, and for that reason, 
probably more important. 

Rewording Hume's text still once again, the thesis of this paper 
may be expressed succinctly in the following way: Without proper evidence, 
alleged facts on which educational policy or practice is based can only 
be classed as fantasies. It is the task of research to convert the 
'right' fantasies into facts and to show the others to be the unsub- 
stantial fantasies they are. 

Before turning to particular examples, I want to say a preliminary 
word about evidence. I have a somewhat skeptical and Bayesian attitude. 
I do not think it is possible to state in mathematically precise terms 
what is to count as evidence and what is not. Evidence also need not 
be collected by systematic exper indentation. The most glorious quanti- 
tative science of them all, namely, astronomy, has scarcely ever been 
able to include exp6riments. Evidence is also not just a matter of quan- 
titative data, organized in obvious quantitative fashion. We do not need 
to perform an experiment or take systematic observations in order to hold 
the firm factual belief that the sun will rise tomorrow. On the other 
hand, when ve turn to the formulation of general principles or general 
ideas about human conduct and how that conduct might be changed by the 
process of education, we must forever be wary and skeptical of those vho 
premise much in general terais and give us principles unsupported by evidence. 
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I fully recognize also that over large periods of time most people 
are indeed taken in by sane unsupported principles. One of the most 
sobering facts I know is that one of the earliest English cliarities 
vas organized to collect money to buy wood to burn witches. The 
record of hman folly committed in the narae of morality or truth is 
too long and dismal to survey here. 

At this point, I turn to sane examples that give a more vivid sense 
of the continued need and the vital place of research in education. I 
consider initially what I call first-order fantasies. These are fantasies 
about general ways of organizing education in matters of theory, policy 
or practice. Afterward I turn to second-order fantasies, which are 
fantasies about methodology or about how we should determine the truth 
or falsity of first-order fantasies. Remember that in the sense I am 
using the terms here, fantasies of either the first or second order can 
be good fantasies in the sense that they can be true. It is the task 
of research to produce the evidence that will convert them from fantasies 
to fact. 

2. First - order Fantasies 

I begin with a classic example of applied linguistics. 

Linguistic fantasies about reading . The importance and significance 
of the work of the American linguist Leonard Bloomf ield is widely recog- 
nized and not challenged by me. The very q.uality of Bloomf ield as a 
linguist makes all the more striking the simplicity of his ideas and 
his apparent total unawareness of the need for data in recommending how 
reading should be taught. His ideas are set forth in the book Let^s Read, 
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A Linguistic Approach (I96I), jointly authored with Clarence L. Barnhart 
and published some years after Bloamf ield ' s death. 

So as not to en'cer into too elaborate an analysis, I shall restrict 
myself to Bloomfield's recommendations about the first materials of 
reading. He enunciates in a few paragraphs (pp. 59-U2) the following. 

1. "Our first material must show each letter in only one phonetic 
value. " 

2. "Our first material should contain no words with silent letters 
(such as knit or gnat) and none with double letters, and none with ccxn- 
binations of letters having a special value (as th in then or be in bean)." 

5. "The letter x cannot be used, because it represents two phonemes 
(ks or £2 ) . " 

"The letter £ cannot be used, because it occurs only in connection 
with an unusual value of the letter u." 

5. "The best selection of values of letters to be used in the first 
materials for reading is the following, " and here follows a large set of 
recctnmendations . 

6. "Our first reading material will consist of two-letter and 
three-letter words in which the letters have the sound values given in 
the above list." 

7. "We should not, at this stage, ask the child to write or print 
the words; that cones much later." 

8. "The early reading lessons should not be very long, for they 
demand a severe intellectual effort. It may be well to take up only two 
words in the first lesson." 

9. "In the second lesson, after review, add two or three more words 
of the same group." 
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10. "The drill should continue until the child can read correctly 
any one of the words yhen the parent or teacher points to it." 

].l. "If the child has learned the pattern in the list of actual 
words, he should be able to read nonsense syllables using the same 
pattern. ... The nonsense syllables are a test of the child's mastery 
of the phoneme. Tell the child the nonsense syllables are parts of real 
words which he will find in the books that he reads." 

12. "The acquisition of nonsense syllables is an important part 
of the task of mastering the reading process. The child will learn the 
patterns of the language more rapidly if you use the nonsense syllables 
in teaching." 

A number of additional principles (pp. 19-U2) are stated, but I 
have given a large enough sample to indicate in an ezcplicit way \jhy 
I label these principles Bloomfield's fantasies . They represent one 
of the purest examples of an analysis of one kind being extrapolated 
and applied to a different kind of problem without recognition of the 
need for data and for evaluation of the correctness of principles in 
their new application. Bloomfield applies some fairly obviaas phonetic 
principles and generalizations, but shows no recognition at any point 
of the need for data to check on the correctness of these principles 
as principles of reading. 

As I use fantasy in this paper, a fantasy can be correct or true, 
but it remains a fantasy unless propar evidence is offered, and this 
is certainly the case for \diat Bloomfield offers us for principles 
about reading. It is almost breathtaking to have him assert, for 
example, the principle that children learn language more rapidly if 



nonsense syllables are used in teaching* To fail to recognize the need 
for data and to state such a learning principle without any systematic 
concept of learning seems^ in our currently skeptical time, almost 
incredible* 

Piaget^s stag es* The influence of Jean Piaget on developmental 
psychology is recognized throughout the world* His very status, however, 
as an imaginative creator of new concepts and theories about children's 
behavior and development has led to an often uncritical acceptance of 
his ideas. I select for emphasis in the present discussion his concept 
of stages of development, which has played such a central part in many 
of his works and has also been ta]'.en over into developmental psycho- 
linguistics* At first glance one might think that the concept of stages 
of development is a methodologically innocent one and scarcely a subject 
of controversy* A perusal of Piaget 's own writings and the large deriva- 
tive corpus soon leads one to another conclusion. As an example I shall 
discuss the analysis of three stages of multiple seria>tion in The Early 
2£2i[th of Logic in the Child by BSi^bel Inhelder and Piaget {l96k). 

To indicate the unequivocal adoption of the idea of stages, I quote 
from page 270: 

We shall distinguish three stages, corresponding to the 
usual three levels. During stage I, there are no seriations 
in the strict sense. The child's constructions are inter- 
mediate between classification and seriation. . . . During 
stage II, there is seriation, but only according to one of 
the criteria, or else the child switches fron one criterion 
to the other. . . . Finally, during stage III (starting at 
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7-8 years), the child re-^ches a multiplicative arrangement 
based on the twofold seriation of the set of elements. 
There is in this discussion, as elsewhere in Piaget, no suggestion that 
matters could be otherwise- -that development could be incremental and 
continuous and that no stages could be identified in nonconventional 
fashion. To adopt the idea of stages as a convenient, conventional way 
of talking in certain restricted contexts is, of course, quite natural.-^ 
It is another thing to talk as if they were real abstractions with a 
verifiable and unequivocal empirical content. 

It is a truism that children develop new capacities and new skills 
as they get older. The problem in evaluating the existence of stages 
is not one of affirming this truism, but rather one of differentiating 
the concept of stages from the equally natural concept of continuous 
development. 

A second related problem that needs detailed study is the extent 
to which the mastery of different concepts follows the same order in 
different children. Again, it is important that the experimental de- 
sign be as meticulous as possible in order not to prejudge the issue. 
It can scarcely be said that Piaget's design in the study I am considering 
satisfies this criterion, and very few others of like nature do either. 

Moreover, if nonconventional plateaus were discovered in the behavior 
of individual children, we would also Mdxit to know whether these plateaus 
r\m across the same set of concepts or occur in a manner that is much 



more randanly related to the concepts themselves and that might reflect 
quite different sorts of processes of maturation in the child. Again, 
little evidence is to be found on this point. 

My first dr£.'.ft of these remarks on the concept of stages received 
an excellent critique irom Harry Beilin, and he has provided me refer- 
ences to his own work and that of Piaget \^iere the question of the actual 
existence of stages is discussed. Piaget writes: 

I now come to the big problem: the problem of the very 
existence of stages; do there exist steps in development or 
is complete continuity observed? . . . when we are faced 
macroscopically with a certain discontinuity we never know 
whether there do not exist sinall transformations which would 
be continuous but which we do not manage to measure on our 
scale of approximation. In other words, continuity would 
depend fundamentally on a question of scale; for a certain 
scale of measurement we obtain discontinuity when with a 
finer scale we should get continuity. Of course this argu- 
ment is quite valiC, because the very manner of defining con- 
tinuity and discontinuity implies that these ideas remain 
fundamentally relative to the sc \le of measurement or ob- 
servation. This, then, is the alternative which confronts 
us: either a basic continuity or else development by steps, 
\Aiich would allow us to speak of stages at least to our scale 
of approximation fl960, p. 121]. 

A more detailed and careful analytical discussion of the concept of 
stage is to be found in Beilin (l97l), and he raises a number of issues 
that are not pertinent to the main point I wish to make here. In the 



printed discussion following Eeilin's paper, there are remarks by Piaget 
that seem to me incorrect, and in general, incorrect for the sane reasons 
that the above quotation about the existence of stages is incorrect. In 
the context of Beilin's paper Piaget (1971, p. 19^) is discussing the 
problem of novelty, and he has this to say: 

Thus, to my regret, I did not find in Beilin's paper 
any reference to this problem of novelty — of the formation of 
novelties in general and the conditions necessary for the de- 
velopment of new structures. ... If there are no novelties, 
then the concept of stages is artificial. There lies the 
\diole problem. 

Before commenting directly on these remarks of Piaget *s, I want to 
compare the almost total absence of serious critical analysis of the 
concept of stages with the controversy in learning theory that existed 
about ten years ago between all-or-none and incremental learning. In 
the latter case the battle was joined with intensity and fruitful re- 
sult because there were strong protagonists on both sides of the argument, 
8Lnd each was determined to establish the incorrectness of the other ^s 
view, and if possible, the correctness of his own view. A large number 
of experiments were performed, and quite detailed analyses were made of 
the data to test \Aether or not the learning of simple concepts or simple 
associations satisfied all-or-none or incremental properties. Analyses 
with special relevance to concept formation in children are to be found 
in Suppes (19^5) and Suppes and Ginsburg (1965)» For the purpose of this 
discussion, the all-or-none model would correspond to Piaget 's idea of 
stages, and the tests of incremental leairning models to the continuous 



-10- 



learning that is the natural alternative to stages. Vfe need the kind of 
sharp exchange and critical examination of experiments and concepts 
characteristic of that learning-theoretic controversy of a decade ago in 
the analysis of the concept of stages in developmental psychology. 

It should be apparent that the attitudes expre.'=3 3ed in the quota- 
tions from Piaget do not provide the basis for this kind of sharp ex- 
change and critical examination. Piaget raises what is essentially 
an irrelevant question of scale. The problem is to find out for the 
^iven scale at which experimentation is conducted whether the process 
is all-or-none or irc:remental, and not whether there are microscales, 
for example, at which the process is continuous even if the data indicate 
all-or-none learning at the ordinary scale of experimentation. The second 
remark about novelty also seems to me to miss the point, especially as 
reflected in the extensive work on all-or-none learning in concept iden- 
tification as opposed to concept formation experiments. It certainly 
is perfectly possible that learning is all-or-none or in terms of stages 
even when no questions of novelty are involved. There is also no reason 
to think that when concept formation and mastery of novel concepts are 
evident that learning is necessarily to be characterized in terms of 
stages rather than incrementally. I say once again that the element 
missing in this discussion by Piaget, and even by Beilin in his other- 
wise excellent article, is the concept of precise and detailed experi- 
mentation with quantitative analysis of data to test for the existence 
of stages. 

Finding out the true state of affairs about stages is important not 
simply for theoretical purposes in developmental psychology, but because 



-11- 



continual use of these concepts is found in the talk of educators in their 
organization of curriculum for young children, in their discussion of the 
skills of young children and in other related ways. It would be easy to 
document the continiial casual reference to Piaget in a variety of educational 
publications in "which teachers are once again being taught dogma without 
data that developmental stages are the way to think about the development 
of children • 

I cannot resist one passing remark on this matter of stages and 
concepts like that of groupement and seriation. The very language used 
by Piaget and his more ardent followers is itself a kind of fantasy of 
mathematics. Those of good faith can believe that back of such talk is 
a real body of concepts that can be put into reasonable order. Those 
of us who are more skeptical face the beginning of the fantasies here 
and wait for a new round of theory and experiments to clear the air. 

A comparative point of intellectual history is perhaps in order. 
There is much about Piaget 's ideas that resembles the kind of suggestive 
^^b woven by Descartes in his principles of philosophy. Descartes, of 
course, was dealing with the physical world and Piaget with the psycho- 
logical world of child development. Both operate in large theoretical 
terms and with little regard for detailed experimental investigation. 
Descartes' tale proved to be enormously seductive in the seventeenth 
century, and even Kewton found it difficult to throw off the Cartesian 
ideas. Leibniz, however, put it correctly when he characterized Descartes' 
physics as a roman de physique , and I shall be bold enough to say that we 
may very likely in the future characterize Piaget 's work as a ranan de 
psychologie s To say this is not to deny that Descartes has occupied an 
important place in the history of physics or that Piaget has occupied an 
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important place in the h-' story of psychology. It is rather to put in 
proper perspective large-scale theories that are as close to fantasies 
as they are to facts. 

Skinner on arithmetic . As an example of a different sort, but at 
the same first-order level of fantasy, I next would like to consider what 
Skinner has to say about teaching arithmetic in his book The Technology 
of Teaching (1968, pp. IU-I5). Here is the opening passage on arithmetic. 
From this excitLng prospect of an advancing science of 
learning, it is a great shock to. turn to that branch of tech- 
nology which is most directly concerned with the learning 
process— education. Let us consider, for example, the teach- 
ing of arithmetic in the lower grades. The school is con- 
cerned with imparting to the child a large nimiber of responses 
of a special sort. The responses are all verbal. They consist 
of speaking and writing certain words, figures, and signs which, 
to put it roughly, refer to nimjbers and to arithmetic operations. 
The first task is to shape these responses— to get the child to 
pronounce and to write responses correctly, but the principal 
task is to bring this behavior under many sorts of stimulus 
control. This is what happens when the child learns to count, 
to recite tables, to count while ticking off the items in an 
assemblage of objects, to respond to spoken or written numbers 
by saying "odd," "even," or "prime." Over and above this 
elaborate repertoire of numerical behavior, most of which 
is often dismissed as the product of rote learning, the teach- 
ing of arithmetic looks forward to those canplex serial arrange- 
ments of responses involved in original mathematical thinking. 



The child must acquire responses of transposing, clearing frac- 
tions, and so on, vdiich modify the order or pattern of the 
original material so that the response called a solution is 
eventually made possible. 

The crudeness of this talk about responses and shaping them without 
serious reference to how arithmetical concepts should be built up is 
typical of this strange and undocumented proposal of how arithmetic 
ought to be taught. The naive and impressionistic character of the 
remarks is attested to by the juxtaposition of the words 'odd', 'even', 
and 'prime' in the middle of the passage. The very special role, for 
example, of students' ever responding prime to spoken or written numbers 
is to be emphasized. The casual way of talking about moving from arith- 
metic to "those complex serial arrangements of responses involved in 
original mathematical thinking" is a reflection of how vague and un- 
substantial Skinner's ideas about the teaching of arithmetic or other 
parts of mathematics are. It would be interesting indeed to have those 
ccanplex serial arrangements of responses made to match any serious piece 
of mathematical instruction, let alone original mathematical thinking. 
The casual talk about acquiring "responses of transposing, clearing 
fractions, and so on" is again indicative of the unthought- out and 
undocumented character of the remarks. 

No evidence is offered about the effectiveness of these ideas for 
the teaching of arithmetic. What is more important— it woiid not even 
be clear fran this passage or the passages that follow how any teacher 
would begin to arrange the complex material of arithmetic in proper 
order for learning by children. It would be interesting to see what 
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Skinner would have to say about the detailed sequence of materials in 
arithmetic, and hov; the proper arrangement of materials should be made, 
according to which principles and on the bas-ls of 'A\at data. It is 
especially ironic to have such a broad and unsubstantiated sketch of how 
arithmetic should be taught without reference to any of the extensive 
literature on the learning of arithmetic. 

I cannot think of a better challenge to Skinnerians than to produce 
a genuine psychological theory of mathematical learning and thinking. 
So far as I know, there is not yet a serio;s contribution from either 
Skinner or his followers on this important educational topic. In some 
quarters at least, I sure the fantasy will remain that sanehow operant 
conditioning is the key to successful mathematical learning. 

5» Second-order Fantasies 

By a second-order fantasy I mean a belief about the efficacy or 
lack of it of a certain methodology^ which :.s unsupported by evidence 
or systematic argument. I begin ^^ath an example much closer to home 
than any I have yet considered and refer to the x^nritings of two authors 
with whan I am in general intellectually sympathetic. 

Campbell and Stanley on experimentation . The fantasy I have in 
mind is the \insupported and yet wholly enthusiastic support of expera- 
mentation by Campbell and Stanley in their well-known chapter on this 
subject in the Handbook of Research on Teaching (1965). As most of 
you will remember, the handbook was itself a product of this association. 
Let me begin with two quotations from the second and third pages of the 
chapter. 
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This chapter is committed to the experiment: as the 
only means for settling disputes regarding educational prac- 
tice, as the only way of verifying educational improvements, 
and as the only way of establishing a cumulative tradition 
in which improvements can be introduced without the danger 
of a faddish discard of old wisdom in favor of inferior 
novelties. . . . even though we recognize experimentation 
as the basic language of proof, as the only decision court 
for disagreement between rival theories, we should not ex- 
pect that "crucial experiments" which pit opposing theories 
will be likely to have clear-cut outcomes. When one finds, 
for example, that competent observers advocate strongly 
divergent points of view, it seems likely on a priori 
. grounds that both have observed something valid, about the 
natural situation, and that both represent a part of the 
truth. The stronger the controversy, the more likely this 
is. Thus we might expect in such cases an experimental 
outcome with mixed results, or with the balance of truth 
varying subtly from experiment to experiment. 

As matters of personal belief, I accept with certain reservations 
what Campbell and Stanley have to say in the quoted passages. V/hat I 
Bid criticizing is the lack of argument for the position, and for this 
reason I have labeled the passage an example of a second-order fantasy. 
The chapter contains no systematic examination of alternatives to ex- 
perimentation, no review of sciences like astronomy which do not engage 



in experainentation in any serious '-;ay and yet achieve remarkable results, 
no attempt to formulate general principles to maJ^:e it clear why experi- 
mentation is so important; in fact, there is no scientifically serious 
attempt to define the concept of an experiment. 

I emphasize that I do not have in mind a rigorous formal treatment 
of the concept of an experiment, but rather a densely argued informal 
consideration of the principles of evidence that offer a systematic 
defense of the use of experimental procedures. For example, within 
a Bayesian framework (I do not mean to suggest that they necessarily 
should adopt such a framework), one can argue that the likelihoods, as 
opposed to the prior distributions of opinion or belief, can be agreed 
upon by different investigators of different theoretical persuasions. 
It is the practical possibility of agreement on likelihood functions 
that makes experimentation attractive. We can of course go on to ask 
the deeper question, why is it that different individuals of quite 
different orientations can agree on likelihood functions and the con- 
ceptual scheme of experimentation when they are far apart theoretically? 
It is not always true that they can so agree, but it is true often enough 
that an analysis can be given of the reasons for agreement in a wide range 
of circumstances. 

My own view would be that the defense should be built on the basis 
of the different status of different kinds of knowledge. We can, for 
instance, agree on how a given group of students answered the items on 
u tost if the test was multiple choice, but we may not be able to agree 
on how to interpret the results, or if we gave an essay test how to 
evaluate even narrowly the essay responses. It seems to me that the 
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defense of experimentation depends heavily on the drawing of such dis- 
tinctions between the kinds of knowledge we have. 

The second major aspect of classical psychological and educational 
experimentation centers around the difficult and elusive concept of 
randomization. Here too, it seems to me that Campbell and Stanley do 
not give the research worker in search of help a detailed and closely 
argued defense of the reasons for randomizing in experiments. If the 
authors felt that the subtle topic of randomization was too difficult 
a one to enter into, clear warnings should have been given the readers 
that they were not attempting any defense of the concept and that it 
was being taken on faith as a wonderful thing. 

I^ter in the chapter there is a section entitled "Some preliminary 
comments on the theory of experimentation," and once again wise remarks 
are made about statistical lore and experimental practice. What is missing, 
however, is that sense of intellectual openness on the one hand and pre- 
cision of argument on the other so very much required in the theory 
of experimentation, or more generally, in applied statistics. Applying 
Hume's dichotomy of having either reasoning about abstract matters or 
evidence about matters of fact, we find that both the quotation above 
and the longer section on the theory of experimentation are neither 
organized around abstract principles from which more principles of ex- 
perimentation are derived, nor validated by a systematic collection of 
empirical evidence bearing on the theory of experimentation. 

It is appropriate to add weight to these general statements by 
some more detailed examples. There are at least three respects in which 
I think the innocent reader might be misled by Campbell and Stanley's 
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generally excellent article. To begin with, the deeper and more varied 
the contact a person has with applied statistics, the more evident it is 
t?.at some experience in seeing the statistical procedures and tests of 
significance derived from first principles is of importance. It is too 
easy for the innocent researcher to divorce in his mind the simple algebraic 
formulation of particular tests or procedures frcxn the probabilistic back- 
ground that justifies their derivation and interpretation. I am not sug- 
gesting something that I think is easy to do within the restrictions 
Campbell and Stanley set for themselves; however, some sense of derivation 
from first principles in at least one example would deepen considerably 
the basis the reader would have for accepting the kinds of distinctions 
introduced. 

My second remark is a more serious and important one. Already at 
the beginning of the nineteenth century, in his treatise on the theory 
of probability, Laplace (182O) emphasized the importance of not simply 
establishing the existence of an effect, but establishing a method for 
estimating the magnitude of an effect. Prora a broad methodological stand- 
point, perhaps the single most important criticism one might make of the 
statistical procedures used and exemplified in the Journal of B>:perimental 
Psychology , methods of which Campbell and Stanley in general approve, is 
the overwhelming use of tests of significance establishing the existence 
of effects, in contrast to the almost total absence of tests that estimate 
magnitudes of effects. 

A simple, but powerful, analysis of such an example is provided by 
Laplace's attempt to estimate the benefits of inoculation for smallpox 
by variolar virus before vaccine was available. (Laplace concluded that 
the mean increased longevity from inoculation was about three years 
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provlded that there was no food shortage or other violent disriiption of 
the enviromert.) Such estimtes of the magnitude of causal effects 
are of the first -iiiiportance in both pure and applied science, and it is 
especially mportant to bring them more to the fore in educational research. 
We may leave to the psychologists aloft in the pares in the Journal of 
Experimental Psycliolo^ry the desicrn of experiments that tesi, for existence 
of effects. In education we are much more ;oncerned with estimating 
magnitudes of effect. If, for example, a new curriculum that costs 
twice as much as an old curriculum produces a measurable effect, but 
that measurable effect is very small in magnitude, then the practical 
use of this curricidum is questionable. 

Mentioning the problem of estimating magnitudes of effects suggests 
ajTimediately broadening the framework of statistical analysis to tliat of 
statistical decision theory. For many educational experiinents, a three- 
fold decision procedure; accept the new procedure of instruction, 
reject it, or continue further experimentation where the ciorrent verdi t 
of nothing yet proved would lead to a new look at oxperlnien';al proccduixs, 
and especially their interpretation. But I shall not attempt to explcre 
these matters further in the present context. 

My thard and final conment on the ''interior analysis" of Campoell 
and Stanley's chapter concerns some remarks they make abouc linear modtlc. 
In discussing tests of significance for time series designs, on page k3 
they assert "Statistical tests would probably involve, in all but the 
most extended time series, linear fits to the data, both for convcnier.ee 
and because more exact fitting v/ould exhaust the de^-rees of freedcn*, 
leaving no opportunity to test the hypothesis of change." 
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It seems to me that here is an exajnple of simplifying too drastical^^ 
and therefore introducing a small-scale kind of fantasy too easily adopted 
by educational researchers. It is a fantasy that we must always test for 
linear relations, because we have no ability to handle nonlinear ones. 
Especially with the use of modern computers, it is almost as easy to deal 
with siinple nonlinear models as linear ones. Exploring the alternatives 
to linearity provides excellent insight into the nature of the relations 
between the variables ai^ does not require necessarily the use of more 
degrees of freedom. I^t us consider, for example, just the simple case 
of two variables, with x the independent variable and y the dependent 
variable. We may express the linear model by the following equation: 

y = a + bx . 

This model has two parameters to be estiinated from the data and thus two 
degrees of freedom are lost. If ve think of the effects of increase in 
X on y proceeding at a faster than linear fashion, we can estimate 
the same number of parameters for the quadratic model: 



2 

y = a + bx . 

On the other hand, if ve think of the nonlinear increase in y with 
increases in x as less than linear, ve can easily test the logarithmic 
model: 

y = a + b log x . 
There is much more to be said about these matters, and I am not 
pretending to give a detailed analysis to complement these brief remarks. 
It is just that in my search for fantasies I have tried to look everywhere, 
even among some of the best established and generally most sensible sources. 
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Chomsky's theory of competence > As a second exainple of a second- 
order fantasy, I select Chomsky's theory of canpetence. If the ideas 
that he seems to be putting forth were correct, they would have sane 
fairly far-reaching implications for educational research and educational 
practice. I classify his remarks quoted below as second order, because 
they recommend an approach to the study of behavior that is at considerable- 
variance with current emphases. The following passage ( Language and Mind , 
1972, pp. 72-73) states Chomsky's methodological point in succinct form. 

Tlie theory oV generative grainmar, both particular and 
universal, points to a conceptual lacuna in psychological theory 
that I believe is worth mentioning* Psychology conceived as 
"behavioral science" has been concerned with behavior and 
acquisition or control of behavior. It has no concept corre- 
sponding to "competence," in the sense in which conpetence is 
characterized by a generative grammar. The theory of learning 
has limited itself to a narrow and surely inadequate concept 
of what is learned- -namely a system of stimulus-response con- 
nections, a network of associations, a repertoire of behavioral 
items, a habit hierarchy, or a system of dispositions to re- 
spond in a particular way under specifiable stimulus conditions. 
Insofar as behavioral psychology has been applied to education 
or therapy, it has correspondingly limited itself to this con- 
cept of "what is learned." But a generative grammar cannot be 
characterized in these terms. What is necessary, in aadition 
to the concept of behavior and learning, is a concept of what 
is learned--a notion of ccmpetence--that lies beyond the 
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conceptual limits of behavior ist psychological theory. Like 
much of modern lin^istics and modern philosophy of language, 
behaviorist psychology has quite consciously accepted method- 
ological restrictions that do not permit the study of systems 
of the necessary ccxnplexity and abstractness. One important 
future contribution of the study of language to general psy- 
chology may be to focus attention on this conceptual gap ard 
to demonstrate hov it may be filled by the elaboration of a 
system of underlying competence in one danain of human intel- 
ligence. 

As in the case of Skinner, the thesis set forth by Chomsky is breath- 
taking in its dogmatic si«iplicity. It could be said that it seems dog- 
matically simple only because I am quoting the introduction of a long 
and coTiplex empirical or theoretical argument. Substantial formal 
arguments and substantial empirical data are offered subsequently, and 
I have distorted the analysis by restricting myself to the quotation 
Just given. Although in the pages that follow, Chomsky amplifies the 
views about competence set forth in this paragraph, he does not ampliiy 
them in a way that satisfies the Humean standards stated at the beginning 
of this lecture. Because a number of psychologists who have influence 
in education have been much impressed by Chcansky's notion of competence, 
it will be useful to examine what he has said and the concept itself in 
somewhat more detail. It is the most elegant of the fantasies I have 
evoked and therefore the appropriate one for final consideration. 

Let me begin with a key sentence of Chansky's remarks that is 
char£.cteristic of conceptual fantasies. After describing the nature 



of behavioral psychology, he says, "but a generative grairimar cannot 
be characterized in these terms." He goes on to say that behavioral 
psychology has accepted methodological restrictions that do not permit 
the study of systems of the appropriate complexity. The fantasy con- 
sists in this negative claim that a generative grammar cannot be charac- 
terized within the framework of behavioral psychology. I have on another 
occasion (Suppes^ I968) criticized a similar claim by Bever, Fodor and 
Garrett (1968), vho attempted to offer what they consider a formal proof 
of the limitations of associationism as a basis for language learning. 
In criticizing their work I characterized it as an example of negative 
dogma as contrasted to negative proof. 

The fantasy claim is especially appropriate in matters of this kind, 
because of the long and classical tradition in mathematics of convertih^ 
negative dogma into negative arguments and establishing thereby a subject 
of much intellectual richness. To transpose the situation slightly, I 
can ijnagine without difficulty the sardonic grin with which a mathematician 
at Alexandria in, let us say, 100 B.C. would have greeted the unsupported 
claim that it is obvious that the trisection of an angle cannot be charac- 
terized in terms of operations performable by a straightedge and compass 
alone. A two-word response would ha.ve been sufficient: Prove it. The 
austerity and precision of negative mathematical arguments are too re- 
straining and perhaps puritanical in their methods for Chomsky and his 
ardent followers. 

To give a negative proof, we must first have a much clearer idea 
of what is meant by the theory of competence than the characterizations 
given by Chcxnsky or his cohorts. If we are talking about language, for 



example, it is strange and wonderful to find only ^rainrar and not semantics 
mentioned in the discussion of canpetence. By example ve are told that 
generative ^raimaars provide a model for theories of canpetence, but what 
is the model of semantic competence? On the one hand, ve are urged not 
to consider arbitrai-y gra.nmars and permit thereby the generation of any 
recursively enumerable set: rather, ve should pick grammars with appro- 
priate restrictions. On the other hand, we are told that it is no part 
of a theory of competence to build in a model of human memory arid per- 
ception and to deal with it in terms of competence ideas. Reflection 
on the passages cited and similar writings by linguists in the Chonsky 
tradition does not cive one confidence that a serious intellectual body 
of ideas is being developed under the heading of the theory of canpetence. 

As my final remark on this, let us even assume that there is such a 
body of serious ideas to be developed. l-Jhile there are certain mathematical 
areas in which one can conceive of formulating what would seem to be a 
theory of competence, or^ is struck by how irrelevant it is to any educa- 
tional or psychological problems. 

The mathem.atical example I vant to deal with is that of mathematical 
proofs. In principle, it is quite straightforward to give a simpleminded 
theory of competence for mathematical proofs; namely, ve know that we can 
formulate within first-order logic almost all cxxrrent mathematical ideas, 
and we can then enumerate the theorems of the subject by enumerating the 
proofs. The enumeration of the proofs will constitute a kind of theory 
of competence. Any proof that exists will eventually turn up in the list 
after only a finite nixmber of predecessors. We have thereby a simple 




algorithm for the prodisctior. of any proof, and ve can show that abstractly, 
sijnply as an ali/orithm, ve can do no better than this. 

lio one thinks that this formal theory of co.-npetence has anything 
serious to do with the psychology of students' discovering elementary 
mathematical proofs in elementary mathematical courses or in mathematicianr 
at work in unknown territory discoverLng new and complex proofs. On the 
one hand, we give a clear and simple theory of competence, one that we 
can state much more about in a sharp mathematical fashion than we can 
in the case of the relation of generative gra:nma>-s to language; yet on 
the other hand, we can aLL recognize at once the essential irrelevance 
of this theory of competence to the psycholc-ical problems of under- 
standing how someone finds a proof or to the educational problem of 
providing instruction to students in giving proofs. 

It seems to me that there is some reason to conjecture that the 
relation of this theory of canpetence for proofs that I have given may 
bear as close a relation to the proper performance theory of proofs as 
does current work on generative grammars, especially with a complex 
transformational component, to correct performance models of language 
usage. In any case, we certainly need something much more definite and 
intellectually precise than Chomsky's historical ruminations on the 
decline and fall of rationalism and its new resurrection under a linguistic 
flag. 

With sane regret, I terminate my remarks on fantasies about competence 
at this point and return to my general theme. 
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Research and the Belief Structures of Education 

I could easily have seized upon a host of lesser targets to provide 
further case material. Because education is of s\:ch universal concern 
in our society, everyone feels free and often competent to speak about 
it in general terms. The body of literature full of unsubstantiated gen- 
eral ideas and principles is now overwhelming. Its authors run fron the 
new romantics like Friedenberg and Holt to a bevy of Journalists turned 
sometime scholars. Characteristic of this literature is the lack of 
intellectual discipline, either in terms of rigorous analysis of gen- 
eral principles or in the presentation of detailed factual evidence to 
support the principles stated. Unfortunately, this kind of literature 
represents nothing new in education. The history of educational change 
is awash with firm prejudices and soggy arguments. I am not, however, 
an advocate of pessimism or skepticism. I think that it is possible to 
improve education, and that research can make an increasingly important 
contribution to this improvement. Let me try to sketch some of the ways 
I think this can happen. 

First of all, it is important to recognize that the belief struc- 
ture of education, the basis on which decisions about policy and practice 
are taken, represents an accretion of many years of experience and fantasy. 
Many of the beliefs are interwoven with other strongly held beliefs about 
how individual, family and societal life should be organized. If nothing 
else, the data of the Coleman report have shown us how difficult it is 
to isolate any particular effects of education from the broad spectrum 
of family and cultural influences. A central problem of research is to 
attack that belief structure where it is unsupported by data or systematic 



is due to the closeness of the subject matter to the layman's own experience. 
The man in the street does not 'expect to be able to give a serious opinion 
about how one should build a better television set or a nuclear fuel plant 
that will reduce pollution. He recognizes, of course, that both of these 
things are worth having, but he seldom has opinions about how they can be 
accomplished. In contrast, ask the mother in the street who has a first 
grader about reading and you are likely to hear some definite views on 
the teaching of reading. 

The nearness of the subject matter is one aspect of the problem, but 
the other aspect, I think, is a problem about the research itself. Even 
a casual scientific inspection of the process the child goes through in 
learning to read quickly demonstrates its complexity. The perceptual, 
cognitive, linguistic and motivational aspects of the process are each 
enormously conplicated, and a detailed conception of how the visual per- 
ception of what the child sees is related to the spoken language he al- 
ready knows is far from available. If we compare this situation to 
the task of improving television sets, the picture is rather dismal. 
The fundamental physics of the processes involved in projecting a tele- 
vised Image on a screen are well understood; many of the fundamental 
concepts go back to classical electromagnetic theory of the nineteenth 
century. 

We have in psychology no ccnparable fundamental theory of perception, 
nor do we have a conparable theory of spoken language comprehension or 
production. At the present time, in solving problems of learning to 
read, we are more in the position of bridge builders before the theory 
of statics was developed than we are in the position of designers of 
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television sets. It is my own view that no matter how beautiful the 
Latin squares of experimental design^ purely empirical studies of dif- 
ferent methods of teaching reading will not solve the problem of giving 
•us the best possible methods, any more than a similarly purely empirical 
approach would ever have led us from the nonelectronic world of I87O to 
the electronic marvels of the 1970s. By this remark I do not mean to 
denigrate the many good empirical studies that have been made of reading, 
but I do wish to put in perspective the severe limitations we face in 
practice in the absence of a deeper running theory of the processes 
involved • 

To build such a theory is a good example of a major relevant problem 
for research in education* Like most research problems in education, the 
solution cannot depend upon the work solely of persons working in educa- 
tion, but rather it must draw upon scientific results from many disciplines, 
in this case ranging from neurophysiology through psychology to linguistics • 
What I consider important as a first step is the recognition that we do 
not have a fundamental theory of the reading process, and in all likeli- 
hood we shall not for some time to come. Let me be a little more explicit 
about what I mean by a fundamental theory of the reading process . I have 
in mind a theory that not only can predict errors or difficulties of stu- 
dents, but a theory that postvilates structures rich enough to process in- 
formation in the same sequence of steps a student does. Put another way, 
the models of the fundamental theory should be complete models of the 
student, and the sense of completeness I use can be given precision by 
using concepts from logic and canputer science* 
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VJhat I have said about reading applies to most other skills and 
subjects taught in our schools and colleges. I emphasize that I mean 
to sound a note of honesty, not of pessimism. Above all, I think the 
time has cane to call for a much deeper theoretical orientation of 
research in education in order thereby to increase its relevance. In 
many areas, ranging from the teaching of reading to the teaching of civics 
the greatest limitation on research is not the absence of hard-data studie 
but the absence of serious and sophisticated theory. Of course, we cannot 
hope to build a mathematical and quantitative theory of educational pro- 
cesses over night. V/e can begin, however, to recognise clearly the ab- 
sence of fundamental theory and to insist on the kind of intellectual 
discipline in the training of our graduate students that will give them 
the tools not merely to make well-designed experimental studies, but to 
construct well-put- together theories that have definite and precise as- 
sumptions and deductive consequences that bear on behavior and the way 
students learn. 

In important ways a good beginning already exists. I would mention 
especially the statistical theory of tests, the theory of measurement, 
some parts of learning theory, and recent economic work on productivity 
in education. Most pressingly needed are mathematical and quantitative 
theories applicable to major areas of curriculimi. In certain areas I 
see the possibility of rapid advance once a cadre of sufficiently so- 
phisticated research workers is available. In elementary-mathematics 
education the well-defined structure of the subject and the long tradi- 
tion of good empirical studies, as well as the modern theory of algorithms 
and abstract machine processes, make available a welter of concepts and 
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intellectual tools for the development of a x'undamental theory of :na the- 
matic s learning and performance at the elementary-school level. To 
some extent, the same is true of second-lane:uage learning, although 
there is not the same tradition of fifty years of careful studies as 
there is in the case of elementary mathematics. Other areas that in- 
volve complex perceptual or cognitive processes are less amenable to any 
direct theoretical attack as yet, and it will undoubtedly be sane time 
before even reasonable looking theories, let alone correct ones, are 
formulated. 

5 . A Research Example from Elementary Mathematics 

I recognize, as does everyone else, that it is much easier to 
criticize than to produce definite constructive results in any area 
of scientific investigation. My original intention was to give as a 
final example of a fantasy some excerpts from my own past writings, 
because the sins of fantasy I have charged others with I have also 
conmitted myself in the past. Even worse, I forecast that I shall 
ccnmiit them again in the futiire. 

After further reflection, I decided it would be more useful, and 
in a deeper sense, expose better my own biases and weaknesses, to sketch 
in a constructive fashion how a precise theoretical attack on problems 
of educational relevance can be made. The curriculum I consider is 
standard and elementary, namely, the algorithms children are taught 
for performing the basic arithmetical operations of addition, subtrac- 
tion, multiplication and division. Also, I first consider performance 
data and only later say something about learning. Since the detailed 
theory of these matters is relatively technical, I have put the formal 
developments in the Appendix. 
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The psychological study of arithmetic skills, like most other parts 
of psychology, has a relatively recent history- -only a few systematic st\idies 
were made before l890* The real impetus was provided by L. Thorndike's 
analysis of the learning of arithmetic in his Educational Psychology (1913, 
191^) and later in his The Psychology of Arithmetic (1922). In an attempt 
to account for the acquisition of arith^ietic skills in terms of his three 
psychological laws— the law of readiness, the lav of exercise and the law 
of effect—he tried to justify and analyse the reason for the traditional 
importance attached to drill and practice in arithmetic; for him the psy- 
chological purpose of drill is to strengthen the bonds between stimuli 
and appropriate responses. He moved on from such fundamental questions 
to the more practical ones of amount and distribution of practice. Ke 
emphasized the advantages of distributed practice and criticized the 
actual distribution of practice in textbooks of his time. Some effects 
of his work on the revisions of textbooks in the 1920s and later are 
documented in Cronbach and Suppes (1969, pp. IO3-IIO). 

In the twenties and thirties there were a large number of good empir- 
ical studies of arithmetic skills, many of which were concerned with de- 
tailed questions that had to be answered in any canplete psychological 
theory of arithmetic. For example, Buckingham (1925) studied student 
preferences and aptitudes for adding up or down in column addition prob- 
lems. An extensive review of this literature may be found in Suppes, 
Jerman and Brian (1968). 

Empirical studies like those of Buckingham were not designed to 
develop an overall theory of arithmetic skills; nor, it is probably fair 
to say, was Thorndike completely sensitive to the gap that existed between 
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his theoretical ideas and the actual algorithms students were ta\:?ht 
to solve problems. There are many stages to work through in developing- 
an adequate theory, and so far as I can see, there is no one point at 
which one can say the theory is now canplete in all respects. If, for 
example, the theory is adequate at some conceptual level of information 
processing, then it is possible to move on to additional perceptual 
questions. Moreover, once a perceptual theory of a certain level of ab- 
straction is successfully developed, it is possible to go on to still 
more detailed perceptual questions, such as requiring the theory to 
include eye movements of students as welj. as their nuimerical responses. 
It is for me an important methodological precept that at no foreseeable 
point shall we reach a fixed and firm bottom beyond which we cannot pro"; 
for further details and a more refined theory. 

I would like to briefly sketch the history of some work of mine and 
my younger collaborators over the past six or seven years. Rather than 
attempt a general coverage, I have decided to select a singular example— 
the simple one of column addition— to illustrate how we have tried con- 
tinually to deepen the theory, and then to discuss what I see as yet 
undone, but practically possible in the near future. 

The data referred to are all taken from our work in computer-assisted 
instruction, but I shall not enter into any of the details. The kinds 
of models discussed can be applied to students vising pencil and paper. 

The first question we tried to answer was how can one predict the 
relative difficulty of different exercises of column addition? If, for 
example, we consider problems up to the size of three columns and three 
vows, we are confronted with approximately one billion problems. A 



meaniiigful theory must drastically reduce this large number of exercises 
to a smaJ.l number of classes in which all members of a class are essen- 
tially the same in difficulty. 

Our firsjt approach (Suppes, Hyman and Jerman, 1967) \ras to identify 
a s»nali nuinber of structural features that would permit ug to apply 
linear regression models to predict either probability of correct re- 
sponse or expected latency of response. Additional applications of 
such regression models may be found in Suppes, Jerman and Brian (1968) 
and Suppes and Morningstar (1972). The application of such regression 
models is exemplified in e.iuation (3) of the Appendix. As can be seen 
froni the infa-mation given there, the fit of the rogrescion model to 
mean student-response data on column addition exercises is not bad. 
Conceptually, however, there are obvious lacunae. The regression model 
that predicts response probabilities does not really postulate a specific 
process by which students apply an algoritkm to solve an exercise. 

The next level of theory developed is aimed precisely at offering 
such process models, models that satisfy the information-processing 
requirements laid down for reading models in the earlier discussion. 
Without doubt, providing an adequate information-processing model for 
column addition is a much simpler affair than providing one for r<sadir.g, 
and I have no illusions about the difference in canplexity. Tho njitural 
theoretical tools for providing process models of algorithmic tasks are 
automata, and for most of elementary arithmetic, simple finite autc-matu 
are satisfactory. There is, however, one weakness In finite auccm&ta 
as ordinarily defined, najnely, they have no place for a probabilistic 
^•heory of error, so the r^tural step is to move from finite detenninistic 
automata to probabilistic automata. 



An automaton becomes probabilistic by making the transition Tunc- 
tion from state to state probabilistic in character. Thus^ from a given 
input and a given internal state there is a probability of goir.r, to any 
one of several different states* In general one wants to make the output 
function probabilistic tili^o. This means that given an internal state 
and an input there is a probability distribution over the next output. 
(These ideas ai-e made formally definite in Definitions 1 and 2 of the 
Appendix.) By drastically reducing the source of error to a small nw- 
ber of parameters, we can develop and apply maii3,geable probabilistic 
automata to student-response data. A detailed example including maximum- 
likelihood estimates of tne three parameters of the automaton are given 
in the Appendix. 

Such a probabilistic automaton model takes a definite step beyond 
a regression model in providing in an abstract sense an adequate 
information-processing model. Prcsn a psychological standpoint, on 
the other liand, the automaton models described in the Appendix are 
unsatisfactory in that they lack any perceptual canponents, and there- 
fore they do not deal directly with how the student actually processes 
the fomat of written symbols in front of him. 

Our current work is very much directed at this point. In prin- 
ciple, it would be possible to continue the development of autcsnaton 
models with an abstract concept of state to represent the student's 
perceptual processing. A weakness of this extension of the automaton 
models is that when the states are left in a general abstract formula- 
tion it is natural to end up designing a different automaton for each 
of the different tasks in elementary mathematics, and a plethora of 



models i-esi:its. Closer exaiuination of the algorithrdc tasks of arith- 
metic facing the student in solving exercises indicates tL,it the vari- 
ous tasks have much in common. This conmonality suggests a somewhat 
different approach, an approach via register machines with perceptual 
instructions. 

Register machines were first introduced by Shepherdson and Sturgis 
(1965) to give a natural representation of computable functions in terms 
chat are closer to the idea of a canputer accepting instructions than to a 
Turing machine. In the case of the representation of computable functiont>, 
a rather simple set of aritlimetic instructions is sufficient. In parti'jular^ 
an unlimited register machine has a denumerable sequence of registers, bvt 
any given progi-am only uses a finite number of these registers and the 
machine accepts six basic instructions: add one to a register, subtract 
one, clear a register, copy from one register to another, and two jmp 
instructions, one conditional and one not. (Tnis set of six instruc- 
tions is not minimal, but it is convenient.) Obviously, for the per- 
ceptual processing that a student does we want a different register 
machine and a radically different set of instructions. In addition, 
it js natural to postulate only a finite fixed number of registers that 
the student can use. 

The basic idea of this approach is to drastically simplify the 
perceptual situation by conceiving each exercise as being presented 
on a grid. The student is represented by a model that has instructions 
for attending to a given square on the grid; for example, in the stan- 
dard algorithms of addition, subtraction and multiplication we begin 
in the upper right-hand corner and then have instructions to move 



dOA-:iward through each column and frar. rl^ht to left across columns • 
Additional insorucoions for scoring the restlts of an opex'ation, for 
outputting the last dirit of a stored numeral, e'.c*, are needed. Somr 
further details are ^iven in the Appendix, but the discussion is not 
as complete as that for autanaton models. 

The basic idea of register machines Is that the different algo- 
rithms are represented by subroutines. One subroutine may be called 
in another, as complex routines are built up. Tlie procedure is famil- 
iar to most of us, even if the langxiage I am usinc is not. For example, 
in performing column multiplication ve use the algorithm of addition, 
vAiich in this case means jallir.r: the subroutine for addition; in long 
division ve call the subroutines for subtraction and multiplication, 
as well as for addition. Each basic subroutine is represented 
by a prograin in terms of the primitive instructions. The problem fron 
a psychological standpoint is to find instructions th^t provide not only 
a realistic description of vhat the student does, a description that can 
be fitted to data in the same way that the autcxnaton models have been 
applied to data, but also a fuller account of how the student processes 
the exercise. 

At the first stage of analyzing register-machine models we can 
get results similar to those for the automaton models by postulating 
error parameters for execution of main subroutines of the routine for 
a given algorithm. More is said about this in the Appendix. However, 
the real purpose of the register machines, in addition to providing 
some explicit analysis of perceptual processing, is to provide a nat- 
lural method for analyzing learning. 



The approach we have adopted is this. At each given stage, the student 
has command of a certain set of subroutines or procedures. To master more 
complex exercises and concepts the student must expand these subroutines or 
iaibed them in more canplex ones. A plausible approach is that the student 
builds up these more complex routines by verbal instruction received from the 
teacher and by interpretation, especially perceptual interpretation, of 
examples. ^:!hen the verbal instruction by the teacher, or say, a computer- 
assisted instruction program, is explicit, and the link to the necessary 
internal instructions is close, a surprisingly simple theory of learn- 
ing within a classical framework can be given. For example, the 
kind of determinate reinforcement for obtaijring finite autanata from 
stimulus-response models, as developed theoretically in Suppes (l9o9), 
can vrithout much modification provide the theory for the buildup of 
the appropriate subroutines. 

I emphasize, however, that we are only beginning the detailed 
analysis of learning in this complex setting, and I am describing the 
conceptual situation. I shall have to wait until later to report on 
the actual empirical accuracy of the learning models we have developed. 
The empirical results obtained with aytcenaton models of perfomance 
have been good enough to encourage us to push on as rapidly as possible 
to the deeper problems of learning. 

In this section I have tried to sketch an example of how one can 
pursue a systematic theory of relevance to education. It should be 
apparent to everyone that the example 1 iiave chosen is exceedingly 
limited, and from many people's standpoint it is an almost trivial 
part of the curriculum. On the other hand, it should be equally 



apparent that the psychological theory of learning and performance in 
a subject matter as simple even as elementary arithmetic is not in it- 
self simple. In fact, a detailed learning theory of elementary arith- 
metic is far more complex than the usual kind of theory psychologists 
consider. One of the problems ve have to face in education is the too 
great willingness of psychologists and others to generalize from quite 
simple tasks to complex ones. 

Vftiat I hope to have brought out in the present discussion, which 
is developed technically in the Appendix, is that the problems of subject- 
matter learning require conceptual developments in their own right that 
do not fall naturally out of general ideas of current psychological 
theories* Yet, with proper use of the variety of conceptual tools 

now available, it does seem possible to provide an^J^ncreasingly adequate 

f 

theory of learning for at least the basic skills, for instance, the 
basic skills of mathematics and language, that constitute a fair 
portion of school curriculum everj'vhere in the world. 

6# Conclusion 

I would like to conclude with a final remark about theory construc- 
tion relevant to er-ication. The times have probably never been so pro- 
pitious for luring some of the ablest young minds into the problems of 
educational research. There exists already a body of methods and results 
of which we can be proud; but it is also clear, especially when ve turn 
to the construction of systematic theories of learning or instruction, 
that we have as yet scarcely scratched the surface. Vfliile we are scratching 
that surface those of us in educational research must impose exacting stan- 
dards not only on ourselves, but also on our neighbors, be they linguists 
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or psychologists, we must demand of them, as well as of ourselves, the 
best possible effort in theory construction. We must above all reject 
the attitude that has in the past sometimes been prevalent that second- 
rate theories and second-rate efforts in. the development of theory will 
suffice for education, and that ve are lucky to get small crumbs from the 
occasional psychologist or linguist or economist who happens to become 
interested in education. We do not need ill- worked- out theories from 
other disciplines. We do not need fantasies of abstractions and plati- 
tudes unsupported by serious and rigorous development. V/hat we need for 
relevance in education are theories of intellectual power and rigor, and 
we should not rest until we get them. 
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Appendix^ 

In this appendix I give seme (but by no means all) of the technical 
details of our research in the psychology of arithmetic. The first 
three sections deal with performance models and the last section deals 
with a learning model. Each section attempts to dig a step deeper than 
its predecessor into the skills of arithmetic. For simplicity I have 
restricted the analysis in this appendix to the simple case of column 
addition, but the methods either already have been or in principle can 
be extended to essentially the entire danain of elementary- school mathe- 
matics (in addition to the references in the main text, see Groen & 
Parkman, 1Q72; Suppes & Groen, 1967)- On the other hand, a good many 
additional developments will be needed to extend this work even to rou- 
tine parts of the undergraduate college mathematics curriculum. (Some 
very empirical first steps at this college level are to be found in 
Goldberg & Suppes, 1972; Kane, 1972; Moloney, 1972.) 

^^"6^^ regression models. As mentioned in the main text I begin 
with regression models that use as independent variables structural 
features of individual arithmetic exercises. I denote the struc- 
tural feature of exercise i in a given set of exercises by f. .. The 
parameters estimated from the data are the values attached to each 
structural feature, (in previous publications ve have referred to these 
structural features as factors, but this can lead to confusion with 
the concept of factor as used in factor analysis.) I denote the 
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4.U 

coefficient assigned to the structural feature by a., and I 

J 

emphasize that the structural featui-es themselves, as opposed to their 
coefficients, are oboectively identifiable by the experimenter in terms 
of the exercises themselves, independent of the response data. 

Let p^ be the observed proportion of correct responses on exer- 
cise i for a given group of students. The natural linear regression 
in terms of the structural features f . . and the coefficients a is 
simply 

p. = Z a.f . . + . 

1 j J ij 0 

Unfortunately, wiien the regression is put in this form, there is no 
guarantee that probability will be pres-^rved as the structural features 
are combined to predict the observed proportion of correct responses. 
To guarantee conservation of probability, it is natural to make the fol- 
lowing transformation and to define a new variable z . . 



1 



1 - p 

(1) z. = log ^ 



Pi 



and then to use as the regression model 

(2) z. = Z a.f. . +0'^ . 

1 j J ij 0 

The numerator of equation (l) contains 1 - rather than p^, so 
that the variable increases monotonically rather than decreases 

monotonically iv-ith the magnitude of the structural features f . .. 

In Chapter 5 of Suppes and Morningstar (1972)^ the following struc- 
tural features were defined for column- addition exercises. 

The feature SUMR is the number of columns in the largest addend. 
For three-rov exercises SVim is defined as I.5 times the number of 
columns, plus .5 if a column sum is 20 or more. For example. 
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a 

STJMR ( + b ) = 1 
c 



a 

sm< ( ^ ) = 

+ c 



1.5 if de < 20 
1 2 if de > 20 



de 

SU^m (ab + c = de) = 2 . 
'Hiis structural feature reflects the number of columns of addition, 
with greater weight being given to columns in three-row exercises than 
in two- row exercises. 

The second structural feature is CAR, which represents the number 
of times the sum of a column, incliiding any na-nbers carried to it. sy.- 
'.:eeds nine. For example, 

a 

CAR ( -i- b ) = 0 
c 

CAR (a + b = cd) = 1 

ab fo if b + d < 9 
CAR ( + cd ) = } 

ef 1 1 if b + d > 9 
ab .J, , 

— (2 If b + d + f>9, a + c + e>9. 
The third structural feature VF reflected the vertical format 
of the exorcise. The vertical exercises with one -digit responses were 
given the value 0. Multicolumn exercises with multidigj.t exercises and 
one-column addition exercises with a response of 11 were given the value,- 
1. One-column addition exercises with a multidigit response other 'h,'.- 
11 v;ere given the value fj. i-'or example. 



ab 

VF ( - cd ) = 0 
e 

abc 

VF ( + def ) = 1 
ghi 

a 

VF ( +_b ) = 5 . 
cd 

This structural feature is meant to reflect the likelihood of the 
mistake of reversing the digits of the correct response, especially in 
a one-column addition exercise. In the conputer-assisted instruction 
environment where students were responding at teletype terminals, re- 
sponses to vertical exercises were typed from right to left, while 
responses to^'^^izontal exercises were typed fron left to right. Thus, 
it was possible for a student to have in mind the correct answer, but/ 
to err by typing the digits in the reverse order. It is fair to say^ 
that this structural feature is of more importance in working at a 
computer-based terminal than \jhen using paper and pencil. 

Table 1 shows a pretest on column addition given to third graders. 
The following regression equation was obtained for the mean response 

Insert Table 1 about here 

data of 65 students taking the test. 

(3) Pi = -53 SUMR. + .93 C/VR. + .51 VF - if. 06 . 

2 

The multiple R was .Ik and R was .^k, vAiich reflects a reasonable 
fit to the data. I shall not enter into further details of the regres- 
sion model, but shall move on to the next level of analysis of these 
same response data. As should be obvious, I am not attempting anything 



i:able 1 

Pretest Exercises in Column Addition 



1) 17 

+ 2 



2) 



o 

C 

+ 5 



8) 



11 

22 



9) 27 
+ 1| 



15) 5261 
+ 28$ 



16) 



75 
+ 25 



5) ih 
+ 15 



10) 



8 

+ 32 



17) 5986 
+ *^755 



6 

+ 15 



5) 565 
+ 21li 



6) kie 
+ 212 



7) 



12 

51 
+ 10 



11) 659 
+ 212 



12) 66 
+ 111 



15) 578 

+ 125 

111) 557 

+ 256 



18) 



20) 



27 
ii6 
+ 88 



19) 7657 
+ 1875 



69 
56 
+ ii8 



like a systenui;:ic pres;entatlcn of data, but cnXy enough tc £':ive a ser.so 
of hov sane of the modeif: do fit. 

Thj^ee - state automaton model . The central weakness of the regressica 
models is that they are not process models. Tixoy do not provide even a 
schematic analysis of the alt'ior j thmic steps the student uses to find ?n 
answer. Automaton models are process models and therefore their use 
represents a v/az^jxal exhonsion of the regression analysis. For the 
exercises in column addition we may restrict oxirseives to finite autom- 
ata, but as ordinarily defined they have no place for errors. However, 
this Is easily introduced by moving from deteiministic state transitions 
to probabilistic ones. 

I begin with the definition of a finite determ.inistlc automaton, 
and then generalize, iliese developments follow Suppes (1969). 

Definition 1. A structure SI = (A,Vj,Vq,M,Q,Sq) _is a finite 
( deterministic ) automaton with output if and only if 
(i) A is a finite, nonempty set, 
(ii) Vj and are finite nonempty sets ( the input and output 

vocabularies 5 respectively ), 

(iii) M is a function from the Cartesian product A X V-. to A 
(M defines the transition table), 

(iv) Q Is a function from the Cartesian product A X V ^ to 
i£ "^he output function), 
(v) Sq _is in A (Sq the initial state ) . 

As an exainple of a finite automaton with ou"cput, that is, a finite 
automaton in the sens'i of this definition, we may characterize an automate 
that will perform two- row colujnn addition. 



V. 



A 



I 



[0,1] , 

{(m,n) : 0 < m, n < 9] , 



V, 



0 



[0,1 



^ . • • , 



9] , 



Q(k, (m,n)) 



M(k, (in,n)) 



(0 if m+n+k<9, 

[1 if m + n + k > 9 , for k = 0,1 , 

(k + m + n) mod 10 , 



s, 



'0 



0 



I 



I 



r 
I 



I ^ 

ER?C I 



Thus the automaton operates by adding first the ones* colmn, storing 
as internal state 0 if there is no carry, 1 if there is a carry, out- 
putting the sum of the ones' column modulus 10, and then moving on to 
the input of the two tens* column digits, etc. The initial internal 
state Sq is 0, because at the beginning of the exercise there is 
no * carry * . 

Definition 2. A structure 21 = <A,Vj,V^,p,q,SQ) a (finite) 
probabilistic automaton if and only if 
( i ) A a finite, nonempty set, 
(ii) Vj and are finite , nonempty sets, 

(iii) p is a function on AXVXA to the interval [0,1] such that for 
each s in A and a in V, p is a probability de nsity over A, i.e., 

(a) for each s* i£ A, p (s*) > 0, 

' ' ' s , cr 

(b) Z p^ (s*) = 1, 
s'€A 

(iv) q is a function on A X X to [0,1] such that for each s in 
^ Q-"^ ^ 1b, ^> ^3 Q. i£ ^ P3:'obability density over 
(v) s^ is in A. 




In the probabilistic generalization of the automaton for column 

addition^ the number of possible parameters that can be introduced is 

uninterestingly large. Each transition M(k, (m^n)) maybe replaced 

by a probabilistic transition 1 - e and , and ea^h 

K^m^n K^m^n 

output Q(k(m,n))^ by 10 probabilities for a total of 2200 parameters. 

A three-parameter automaton model structurally rather close to 
the regression model is easily defined. First^ two parameters, € 
and 1], are introduced according to whether there is a * carry* to the 
next column. 

P(M(k, (m,n)) = 0|k + m+ n<9)=l-G 

and 

P(M(k, (m,n)) = l|k + m+ n>9)=l-Tl. 
In other words, if there is no 'carry', the probability of a correct 
transition is 1 - g and if there is a 'carry' the probability of 
such a transition is 1 - T|. The third parameter, is simply 

the probability of an output error. Conversely, the probability of 
a correct output is: 

P(Q(k, (m,n)) = (k + m + n) mod 10) = 1 - Y • 
Consider now exercise i with C^ carryg and digits . If 

we ignore the probability of two errors leading to a correct response 
(e.g., a transition error followed by an output error), then the proba- 
bility of a correct answer is just 

C. D.-C.-l 

(h) P(Correct Answer to Exercise i) = (l - y) (l - '0) ^(1 - e) ^ ^ 

As already indicated, it is important to realize that this equation is 
an approximation of the 'true' probability. However, to ccsnpute the 
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exact probability it is necessary to make a definite assumption about 
how the probability y of an output error is distributed among the 
nine possible wrong responses. A sample and intuitively appealing 
one-parameter model is the one that arranges the 10 digits on a circle 
in natural order with 9 next to 0, and then makes the probability of 
an error j steps to the right or left of the correct response 6^. 
For example, if 5 is the correct digit, then the probability of re- 
sponding k is 6, of 5 is 6^ of 2 is 6^, of 1 is (>^, of 0 is 
6 , of 6 is 6, of 7 is 6 , etc. Thus in tenns of the original 
model 

Y = 2(6 + 6^ + 6^ + 6^) + 6^ • 
Consider now the exercise 

hi 

Then, where d^ = the i digit response, 

P(d^ = 2) = (1 - y) , 
P(d2 = 6) = (1 - y)(i . + . 

Here the additional term is 716, because if the state entered is 0 
rather than 1 when the pair (7,5) is input, the only way of obtaining 
a correct answer is for 6 to be given as the stim of 0 + 4 + i, ^hich 
has a probability 6. Thus the probability of a correct response to 
this exercise is (l - y)[(1 - y)(1 " T]) + T16]. Hereafter we shall 
ignore the T]6 (or €6) terms. 

Returning to (k) we may get a direct canparison with the linear 
regression model defined by (5), if we take the logarithm of both sides 
to obtain: 
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(5) log p. = D. log (1 - y) + C. log (1 - Tl) + (D. - C. - 1) log (1 - c) 

and estUnate log 1 - log 1 - T\, and log 1 ^ € by regression with 
the additive constant set equal to zero. We also my use sane other ap- 
proach to estimation such as minimum x or maxMum likelihood. An 
analytic solution of the standard maximum-likelihood equations is 
difficult, but the maximum of the likelihood function can be found 
numerically. 

The automaton model natm^ally suggests a more detailed anal^'sis of 
the data. Unlike the regression model, the automaton provides an immedi- 
ate analysis of the digit-by-digit responses* Ignoring the eo-type terms, 
we can in fact find the general maximum-likelihood estimates of a, 
and T] when the response data are given in this more explicit foim. 

Let there be n digit responses in a block of exercises* For 
1 < i < n let x^ be the raxidom variable that assumes the value 1 
if the i^'^ response is correct and 0 otherwise. It is then easy to 
see that 

r 

- y) if i is a ones '-column digit. 



P(x^ = 1) = , 



(l - y)(1 - e) if it is not a ones' column and there 
is no carry to the i'^^ digit, 

(1 - y)(1 - T]) if there is a carry to the i^^ digit, 

granted that e6-type terms are ignored. Similarly for the same three 
alternatives 



P(x. = 0) = 
1 



1 - (1 - y)(i - e) 
1 - (1 - y)(l - T]) 
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So for a string of actual digit responses x we can vi'ite the 

1 n 

likelihood function as: 

(6) t'U^,...,xJ := (l-Y)V(l-c)''(l-T]rtl- (1-y)(1-0]^[1-(1-y)(1"1|)1^ , 

where a = numter of correct responses, b = number of incorrect responses 
in the ones' column, c = number of correct responses not in the ones' 
colxjmn when the intern^ state is 0, d = number of correct responses 
when the internal rtate is 1, e = number of incorrect responses not 
in the ones' column when the intern^ state is 0, and f = number of 
incorrect responses ^en the internal state is 1. (in the model sta- 
tistical independence of responses is assured by the correction pro- 
cedure.) It is more convenient to estimate = 1 - = 1 - 
and T]' =1-1]. I-laking this change, taking the logarithm of both sides of 
(6) and differentiating with respect to each of the variables, we ob- 
tain three equations that determine the ma>:imum-likelihooa estimates 
of and T]': 

^ _a_ b eg' fll^ 

^ y' " 1 - " 1 - Y'e' " 1 " y'^' " ' 

5L c_ ey' _ ^ 

^ " €' " 1 - y'e' " ^ ' 

dTV 11' 1 - y'l\' ^ • 

Solving these equations, ^e obtain as estimates: 

f ^ a - c - d 
^ a + b- c- d ^ 



i:,Soiinaoes the parameters for -iv- /.-r-r.^ third- -rade data aireadv- described, 
as veil as a .3raph of rhe orse-ved ^^\d preJioteS response probabilities 
for the exercises sho-.r, i:. Tallo I, are riven in :3iapter h of Suppes and 
Morningstar (1972). (This chapiter iv-ac vr-ivten in collaboration with 
Alex Cannara and he ir. respcnsltle fcr th-^ dci-;a ar-alysis.) The esti- 
inates are: y = -Oh^i, ^ .'Xfc5 and ,057-'. The graph of 
response probabilities ic reproduced as Figure 1. A detailed discussion 

Insert Fi,:itre 1 acc-jst here 

of the fit of the model and fia-ther analysis of some of the discrepancies 
are to be found in the chapter mentioned. Here I liave tried cnly to give 
a sense of how this kind of model can be brought into direct confronta- 
tion vith data. 

Register machines vith perceptual instructions . To introduce 
greater generality ar:d to deepen the analysis to include specific Ideas 
about the perceptual processing of a oolumn-addition exercise, I move 
on to register machines for the reasons described in Section h of the 
main text. This research is being conducted in collaboration with 
Lindsay L. Flannery. 

For column addition three registers suffice in our scheme of analysis. 
First there is the stimulus-supported rerjister [SS] that holds an encoded 
representation of a printed symbol to which the student is perceptually 
attending. In the present case the alphabet of such symbols consists 
of the 10 digits and the underline symbol 'J . As a new symbol is at- 
tended to, previously stored si'mbols are lost unless transferred to a 
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EXERCISE 



Fig. 1. Predicted and observed probability correct for 5-parameter 
automaton model. 



non-stimulus-supported register. The second register is rhe non- stimulus - 
supported register [NSS]. It provides long-term storage for computational 
results. The third register is the operations register [OP] that acts as 
a short-term store, both for encodings of external stimuli and for restdts 
of calculations carried out on the contents of other registers. It is also 
primarily non-stimulus-supported. 

As already stated in the main text, we drastically simplify the per- 
(ieptual situation by conceiving each exercise as being presented on a 
grid with at most one symbol in each square of the grid. For colmn 
addition we number the coordinates of the grid from the upper right-hand 
corner. Thus, in the exercise 

15 

2k 

the coordinates of the digit 5 are (1,1), the coordinates of h are (2,1), 
the coordinates of 7 are (3^1)^ the coordinates of 1 are (1,2) and so 
forth, with the first coordinate being the row nxxmber and the second 
being the column number. 

The restricted set of instructions we need for column addition are 
the following 10. 

Attend (a,b): Direct attention to grid position (a,b). 
(±a, ib): Shift attention on the grid by (±a, ±b). 
Readin [SS]: Read into the stimulus-supported register 
the physical symbol in "J:he grid position 
addressed by Attend. 
Lookup [Rl] + [R2]: Look up table of basic addition facts for 

adding concents of register [Rl] and [R2] 
and store the result in [Rl], 
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Copy [Rl] in [R2]: Copy the content of register [Rl] in 

register [R2]. 

Deleteright [R]: Delete the rightmost symbol of register [R] . 

" Jump L: Jump to line labeled L. 
Jump (val) R,L: Jump to line labeled L if content of register [R] 
is val. 

Outright [R]: Write (output) the rightmost symbol of 
register [R] at grid position addressed 
by Attend. 

End: Terminate processing of current exercise. 
Exit: Terminate subroutine processing and return 
to next line of main program. 

Of the 10 instructions only Lookup does not have an elementary character. 
In our complete analysis it has the status of a subroutine built up fron 
more primitive operations such as those of counting. It is, of course, 
more than a problem of constructing the table of basic addition facts 
from counting subroutines; it is also a matter of being able to add a 
single digit to any number stored in the non- stimulus- supported register 
[NSS] or [OP], as, for example, in adding many rows of digits in a given 
column. I omit the details of building up this subroutine. 

It should also be obvious that the remaining nine instructions are 
not a minimal set; for example, the unconditional jump instruction is 
easily eliminated. We do think the nine are both elementary and psy- 
chologically intuitive for the subject matter at hand. 

To illustrate in a simple way the use of subroutines, we may con- 
sider two that are useful in witing the program for column addition. 



-56- 



The first is the vertical scan subroutine, which is needed for the fol- 
lowing pui'pose. In adding rows of numbers with an uneven number of 
digits, we cannot simply stop when we reach a blank grid square on the 
left of the topmost row. We must also scan downward to see if there 
are digits in that column in any other row. A second aspect of this 
same problem is that in our model the student is perceptually ^^ocessing 
only one grid square at a time, so that he must have a check for finding 
the bottom row by looking continually for an underline symbol. Otherwise 
he could, according to an apparently natural subroutine, proceed indefi- 
nitely far downward encountering only blanks and leaving entirely the 
immediate perceptual region of the formatted exercise. Here is the sub- 
routine. In the main program it is preceded by an Attend instruction. 

Vertical Scan Subroutine 

V-scan (0-9, _ ) 

Rd Readin 

Jmp (0-9, _ ) SS, Fin 
Attend (+1,-1) 
Readin 

Jump ( _ ) SS, Fin 
Attend (+0,+l) 
Jump Rd 
Fin Exit 



The labels Rd and Fin of two of the lines are shown on the left. 



The second subroutine is one that outputs all the digits in a register 
working from right to left. For example^ in column addition, after the 
leftmost column has been added, there may still be several digits remain- 
ing to print out to the left of this column in the ^answer* row. 

Output [R] 

Put Outright [R] 

Deleteright [R] 
Jump (0-9) R, Put 
Exit 

Using these two subroutines the program for vertical addition is relatively- 
straightforward and requires 26 lines. I number the lines for later ref- 
erence; they are not a part of the program. 

Vertical Addition 



1. Attend (l,l) 

2. Readin 

5. Copy [SS] in [OP] 

k. Attend (+l,+0) 

5. Readin 

6. Opr Lookup [OP] + [SS] 

7. Rd Attend (+1,0) 

8. Readin 

9. Jump (0-9) SS, Opr 

10. Jump (Blank) SS, Rd 

11. Attend (+1,0) 
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12. Outright [OP] 

13. Deleteright [OP] 
1^. Copy [OP] in [NSS] 

15. Attend (l,+l) 

16. V-scan (0-9, _ ) 
IT. Jump ( _ ) SS, Fin 

18. Jump (0-9) SS, Car 

19. Copy [SS] in [OP] 

20. Jump Rd 

21. Car Copy [NSS] in [OP] 

22. Jump Opr 

23. Fin Jump (Blank) NSS, Out 
2h. Attend (+1,0) 

25. Output [NSS] 

26. Out End 



To show how the program works, we may consider a simple one-column additi 
exercise. I show at the right of each line the content of each register 
just before the next row is attended to, i.e., after all operations have 
been performed. 
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1; 

5 
3 
8 



[SS] 
k 
5 
5 
8 



[OP] 

k 

9 
12 
20 
20 



[NSS] 



This kind of analysis can be generalized to prove that the program is 
correct, i.e., will output the correct answer to any column- add it ion 
exercise, but this aspect of matters will not be pursued further here. 

By attaching error parameters to various segments of the program, 
performance models are easily generated. For comparative purposes we 
may define a performance model essentially identical to the two-state 
probabilistic automaton already introduced for column addition restricted 
to two rows. To lines 6-12 we attach the output error parameter 
to lines 13-19 we attach the 'carry* error parameter 1] if there is a 
carry, and the error parameter e if there is not. Given this charac- 
terization of the error parameters the two performance models are bs- 
haviorally identical. On the other hand, it is clear that the program 
for the three-register machine is much more general than the two-state 
probabilistic automaton, since it is able to solve any vertical addition 
exercise. It is slLso obvious that other performance models can easily be 
defined for vertical addition by introducing error parameters attached to 
different segments of the program. 
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Leaming. In an earlier article (Suppes, I969), I proved that 
given any connected finite automaton there is a stimulus-response model of 
learning that is asymptotically iscmorphic to the autariaton, i.e., as the 
number of trials approaches infinity, and initially all stimuli may be 
unconditioned to any of the desired responses. In one clear sense, how- 
ever, the theorem proved is too weak because of the special character of 
the reinforcement schedule. KOiat is required is reinforcement of the 
transitions from each response-stimulus pair to the next response, where 
the responses, internal or external, constitute the states of the automaton. 
The response on trial n must become conditioned to the pair consisting 
of the response of trial n - 1 and the stimulus on trial n. A canplete 
matching of the reinforcement schedule to such conditioning connections 
is often not experimentally feasible. 

At the other end of the scale, Rottmayer (1970) proved the following 
theorem. I^t be a classification scheme for dividing a possibly 
infinite set of stimuli or stimulus patterns into two classes, such that 
the classification of any pattern can be accomplished by a finite automaton. 
Then there is a stimulus-response model that can learn the classification 
scheme Q, given as reinforcement only the information of whether its 
classification of successively presented patterns is correct or incorrect. 
The weakness of this theorem is that the learning is very slow, and 
machinery for building up a hierarchy of concepts is not directly provided. 

The shift fron automata to register machines seems promising not only 
for the development of performance models, but also for the construction 
of learning models. learning in this framework consists of building 
internal programs of increasing complexity. The reinforcement procedures 
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realistically fall between the two extremes described above. Verbal 
directions and corrections correspond closely, but not exactly, to 
segments of an appropriate program (I emphasize an because the internal 
program constructed is not necessarily unique). 

I restrict myself here to an example of this approach. I take as 
the class of exercises single-column addition, but with an indefinite 
number of rows. The program is simpler than the general one given 
above, and it is easy to see the relation between what is said to the 
student by the teacher or computer to the desired internal program. 
In Figure 2 I show the verbal instructions on the right with the physical 

Insert Figure 2 about here 

pointing to the relevant part of the displayed exercise indicated in 
parentheses. When errors are made, still more detailed instructions, 
tailoj-ed to the particular error, can be given, but I do not consider 
such error messages here. 

In Figure 2, learning parameters c^, c^, c^ and C|^ are shown 
for the four segments of the program. Various learning models can be 
formulated in terms of these fovir parameters. The simplest is the one 
that assumes independence of the four parts. If we treat the probability 
of successive errors combining to yield a correct response as having 
probability zero, then the mean probability for a correct response on 
trial n for the independence model is simply: 

h 

P (Correct Response) = 1^ (l - (l - c.)"*""^) 

c=l ^ ' * 
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Internal Program 

Attend (l,l) 
Readin 

Transfer [SS] to [OP] 
Attend (+1,0) 
Readin 

Lookup [OP] + [SS] 

Attend (+1,0) 
Readin 

Jump (0-9) SS, Opr 

Attend (+1,0) 
Output [OP] 
End 



Verbal Instructions 



c. Start here (pointing). 



c Add first two digits (pointing) 



Now add again (pointing) (if conditional 
jump satisfied) 
c or 

Notice end of colvimn (pointing at ) 
(if conditional juimp not satisfiedX" 



c^ Write answer here (pointing) 



Fig. 2. Single-column addition. 



At the other exvremG a hierarchical model postulates that the i^^ 
segment of the prograin cannot be learned \:ntil the i-l^^ segjnent is 
learned. This hierarchical model leads to the following transition 
matrix, where state 0 represents all segments as unlearned, state 1 
represents the first set^tnent only as learned, etc. 





h 


5 


2 


1 


0 


k 


1 


0 


0 


0 


0 


3 






0 


0 


0 


2 


0 






0 


0 


1 


0 


0 


=2 


^-2 


0 


0 


0 


0 


0 


=1 





Detailed comparison of these two models, especially for testing against 
data, requires considerable further development, but the relevant mathe- 
matica3. and probabilistic techniques are familiar in the literature of 
mathematical learning theory. 

What is missing fron a theoretical standpoint is a deeper conceptuali- 
zation of the relation between verbal instructions and reinforcements on 
the one hand and the construction of appropriate segments of internal 
programs on the other. In the example given above, the crucial concept 
of iteration or recursion embodied in the conditional jump instruction is 
presumed to be learned fron the instruction "Now add again," with again 
mainly carrying the force of the recursion. I hope to have something more 
to say in the near future about this difficult and important problem. 
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Footnotes 

■'■Roger BrowTi puts the inatter nicely, "... one naturally falls 
into the habit of speaking of stage I and stage II and so on. There is 
no harm in that so long as we recognize that these are imposed stages, 
laid upon continuous data by the ihvestigator as an analytic convenience 
[ Psycholinguistics ^ 1970, p. 100]." 

^The research reported in the Appendix has been supported by the 
National Science Foundation Grant NSFGJ-UUjX and U. S. Office of Education 
Grant OBG- 970-002^(057). 
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